Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

A Formula for the Capacity of the General Gel'fand-Pinsker Channel

We consider the Gel'fand-Pinsker problem in which the channel and state are general, i.e., possibly non-stationary, non-memoryless and non-ergodic. Using the information spectrum method and a non-trivial modification of the piggyback coding lemma by Wyner, we prove that the capacity can be expressed as an optimization over the difference of a spectral inf- and a spectral sup-mutual information rate. We consider various specializations including the case where the channel and state are memoryless but not necessarily stationary.Comment: Accepted to the IEEE Transactions on Communication

Convolutional Neural Network on Three Orthogonal Planes for Dynamic Texture Classification

Dynamic Textures (DTs) are sequences of images of moving scenes that exhibit certain stationarity properties in time such as smoke, vegetation and fire. The analysis of DT is important for recognition, segmentation, synthesis or retrieval for a range of applications including surveillance, medical imaging and remote sensing. Deep learning methods have shown impressive results and are now the new state of the art for a wide range of computer vision tasks including image and video recognition and segmentation. In particular, Convolutional Neural Networks (CNNs) have recently proven to be well suited for texture analysis with a design similar to a filter bank approach. In this paper, we develop a new approach to DT analysis based on a CNN method applied on three orthogonal planes x y , xt and y t . We train CNNs on spatial frames and temporal slices extracted from the DT sequences and combine their outputs to obtain a competitive DT classifier. Our results on a wide range of commonly used DT classification benchmark datasets prove the robustness of our approach. Significant improvement of the state of the art is shown on the larger datasets.Comment: 19 pages, 10 figure

Quantum mechanical calculations of rotational-vibrational scattering in homonuclear diatom-atom collisions

Most calculations of the vibrational scattering of diatom-atom collisions use the breathing sphere approximation (BSA) of orientation averaging the intermolecular potential. The resulting angularly symmetric potential can not cause rotational scattering. We determine the error introduced by the BSA into observables of the vibrational scattering of low-energy homonuclear diatom-atom collisions by comparing two quantum mechanical calculations, one with the BSA and the other with the full angularly asymmetric intermolecular potential. For ·reasons of economy the rotational scattering of the second calculation is restricted by the use of special incomplete channel sets in the expansion of the scattering wavefunction. Three representative collision systems are studied: H_2-Ar, O_2-He, and I_2-He. From our calculations, we reach two conclusions. First, the BSA can be used to analyze accurately experimental measurements of vibrational scattering. Second, measurements most sensitive to the symmetric part of the intermolecular potential are, in order, elastic cross sections, inelastic cross sections, and inelastic differential cross sections. Elastic differential cross sections are sensitive to the potential only if the collision is "sticky," with scattering over a wide range of angles; I_2-He is such a collision. Otherwise the potential sensitivity of elastic differential cross sections is concentrated in the experimentally difficult region of very small angle scattering